Using Many-Facet Rasch Measurement and Generalizability Theory to Explore Rater Effects for Direct Behavior Rating--Multi-Item Scales

Autor/inn/en	Anthony, Christopher J.; Styck, Kara M.; Volpe, Robert J.; Robert, Christopher R.
Titel	Using Many-Facet Rasch Measurement and Generalizability Theory to Explore Rater Effects for Direct Behavior Rating--Multi-Item Scales
Quelle	In: School Psychology, 38 (2023) 2, S.119-128 (10 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Anthony, Christopher J.) ORCID (Styck, Kara M.) ORCID (Volpe, Robert J.) ORCID (Robert, Christopher R.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	2578-4218
DOI	10.1037/spq0000518
Schlagwörter	Item Response Theory; Generalizability Theory; Interrater Reliability; Behavior Rating Scales; Elementary School Students; Student Motivation; Learner Engagement; Behavior Problems + Suchen Sie Ihr Suchwort? Item-Response-Theorie; Interrater-Reliabilität; Schulische Motivation
Abstract	Although originally conceived of as a marriage of direct behavioral observation and indirect behavior rating scales, recent research has indicated that Direct Behavior Ratings (DBRs) are affected by rater idiosyncrasies (rater effects) similar to other indirect forms of behavioral assessment. Most of this research has been conducted using generalizability theory (GT), yet another approach, many-facet Rasch measurement (MFRM), has recently been utilized to illuminate the previously opaque nature of these rater idiosyncrasies. The purpose of this study was to utilize both approaches (GT and MFRM) to consider rater effects with 126 second- through fifth-grade students who were rated on two DBR-Multi-Item Scales by four raters (22 of these ratings were fully crossed). Results indicated the presence of rater effects and revealed nuances about their nature, including showing differences across construct domains, identifying items that are potentially more susceptible to rater effects than others, and isolating specific raters who appear to have been more susceptible to rater effects than other raters. These findings further indicate the indirect nature of DBRs and offer potential avenues for addressing and ameliorating rater effects in research and practice. (As Provided).
Anmerkungen	American Psychological Association. Journals Department, 750 First Street NE, Washington, DC 20002. Tel: 800-374-2721; Tel: 202-336-5510; Fax: 202-336-5502; e-mail: order@apa.org; Web site: http://www.apa.org
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01